Papers with exact match score
SKILL: Structured Knowledge Infusion for Large Language Models (2022.naacl-main)
Copied to clipboard
| Challenge: | Large language models (LLMs) have demonstrated human-level performance on a vast spectrum of natural language tasks. |
| Approach: | They propose a method to infuse structured knowledge into large language models by directly training T5 models on factual triples of knowledge graphs (KGs). |
| Outcome: | The proposed method outperforms baseline models on FreebaseQA and WikiHop, as well as the Wikidata-answerable subset of TriviaQA and NaturalQuestions. |
REANO: Optimising Retrieval-Augmented Reader Models through Knowledge Graph Generation (2024.acl-long)
Copied to clipboard
| Challenge: | Existing knowledge graphs suffer from incompleteness and lack information critical for answering given questions. |
| Approach: | They propose to enhance the open domain question answering model with a knowledge graph generation module that generates KGs from the passages and an answer predictor. |
| Outcome: | The proposed model improves the exact match score by 2.7% on the EntityQuestion dataset, with an average improvement of 1.8% across all the datasets. |
COMMIT: Code-Mixing English-Centric Large Language Model for Multilingual Instruction Tuning (2024.findings-naacl)
Copied to clipboard
| Challenge: | COMMIT improves the exact match score of low-resource language QA by 32x. |
| Approach: | They propose to specialize instruction tuning to deviate from English-centric instruction tuning . they propose to perform cross-lingual alignment to overcome data imbalance . |
| Outcome: | The proposed method improves the exact match score of low-resource language QA by 32x. |
What’s in a Name? Answer Equivalence For Open-Domain Question Answering (2021.emnlp-main)
Copied to clipboard
| Challenge: | A flaw in QA evaluation is that annotations often only provide one answer . therefore, model predictions semantically equivalent to the answer but superficially different are considered incorrect. |
| Approach: | They explore using alias entities from knowledge bases to extract additional answers . they incorporate additional answers for evaluation and model training with equivalent answers based on the results . |
| Outcome: | The proposed solution improves the accuracy of evaluation with additional answers and improves model training with equivalent answers. |